Skip to content

[ENH] BEP036 - Phenotypic Data Guidelines#2123

Open
ericearl wants to merge 97 commits into
bids-standard:masterfrom
surchs:bep036-patch-mkdocs-macro
Open

[ENH] BEP036 - Phenotypic Data Guidelines#2123
ericearl wants to merge 97 commits into
bids-standard:masterfrom
surchs:bep036-patch-mkdocs-macro

Conversation

@ericearl

@ericearl ericearl commented May 30, 2025

Copy link
Copy Markdown
Collaborator

The BEP leads can meet as-needed to discuss this BEP PR

Coordinate a meeting by emailing Eric Earl: eric.earl.nih@gmail.com.

Communicate on this PR to provide feedback otherwise.

HTML preview of this BEP

BEP036 brings guidelines for best tabular phenotypic data to the BIDS specification.

  • Includes an appendix called phenotype.md
  • Includes a new AdditionalValidation key for the dataset_description.json, for which the usage is described in the modality agnostic files sections
  • Includes the new option to store session_id as the second column in the participants.tsv

Additional Links

  1. Original Google Doc
  2. Draft BIDS Validator errors and warnings
  3. BIDS Examples PR

Co-authored-by: Eric Earl eric.earl@nih.gov @ericearl
Co-authored-by: Samuel Guay samuel.guay@umontreal.ca @SamGuay
Co-authored-by: Sebastian Urchs sebastian.urchs@mcgill.ca @surchs
Co-authored-by: Arshitha Basavaraj arshitha.basavaraj@iiitb.ac.in @Arshitha

ericearl and others added 4 commits May 20, 2025 08:24
Quick update before merging our PR on surchs fork
BEP036 brings guidelines for best tabular phenotypic data to the BIDS specification.

- Includes an appendix called `phenotype.md`
- Includes admonitions for the guidelines in-line with modality agnostic files sections

---------

Co-authored-by: Eric Earl <eric.earl@nih.gov>
Co-authored-by: Samuel Guay <samuel.guay@umontreal.ca>
Co-authored-by: Sebastian Urchs <sebastian.urchs@mcgill.ca>
Co-authored-by: Arshitha B <arshitha.basavaraj@iiitb.ac.in>
Changed "e.g." to "for example" to follow contributing style guidelines.
Comment thread src/modality-agnostic-files/data-summary-files.md
Comment thread src/modality-agnostic-files/data-summary-files.md Outdated
Comment thread src/appendices/phenotype.md Outdated
each `phenotype/<measurement_tool_name>.json` data dictionary.
This improves reusability and provides clarity about the measurement tool.

### 5. Use the demographics file for common variables about participants

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying from https://github.com/surchs/bids-specification/pull/1/files#r2103117486

For this section, would it make sense to suggest that demo-like information be prioritized in this file rather than participants.tsv, making the latter primarily a list of subject IDs? I haven't seen this explicitly addressed anywhere, though I'm unsure if it's something we want to formalize 😬
Something like this could follow the paragraph?:

When all demographic data is stored in phenotype/demographics.tsv, participants.tsv may serve primarily as a minimal listing of subject identifiers with only the participant_id column.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. It'd be good to mention this.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I am new to commenting a PR and also relatively new to BIDS - so my comment might be stupid.

However, I want to draw attention to a potential issue with the handling repeated usage of a phenotype/ measurement. Currently, if a measurement is conducted more than once, a run_id column MUST be added that "corresponds to an existing run- entity used in a filename(s)" (https://bids-specification--2123.org.readthedocs.build/en/2123/modality-agnostic-files/phenotypic-and-assessment-data.html).

However, in my case (and I guess many others), a phenotype measurement tool might be conducted independently of a run in an experimental task. For example, I conduct a mood questionnaire 7 times over the course of one session in which multiple tasks (with multiple runs) are conducted. Thus, I think it is not reasonable that the run_id column in the phenotype file corresponds to an existing run_ in a (task) filename. Instead, something like a measurement_timepoint_id (or similar) that is not related to the run_ in other files might be a good idea. Again, I am sorry if my comment is stupid or I misunderstood something.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Along this line: The current proposal states:

A participant identifier of the form sub-, matching a participant entity found in the dataset. Note that data for one participant MAY be represented across multiple rows in case of multiple sessions or runs, and therefore the entry in the participant_id column will be repeated. The combination of participant_id, session_id and run_id MUST be unique.

The problem with this is that it doesn't cover the following cases which are in many datasets that I am familiar with:

Example 1: A dataset in which there are multiple training sessions on some equipment or on some test procedure on multiple different days before any imaging or any data that would have a run occurs.

Example 2: Sleep questionnaires and other questionnaires that administered multiple times un-associated with any imaging sessions.

Both of these require another column identifier (not associated with the run).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VisLab Great to hear from you! Thanks for reading the BEP. Now to your comments...

Example 1: A dataset in which there are multiple training sessions on some equipment or on some test procedure on multiple different days before any imaging or any data that would have a run occurs.

We may not have made it explicit, but here is this example in the HTML preview of the appendix. It covers what to do when there's "1 participant with 2 sessions, where 1 session is only tabular phenotype and the other is only imaging". The same idea extends to more than 2 sessions of some combination of acquired data and tabular phenotypic data. For instance, in your Example 1, we could say:

Define many phenotypic session names

  • ses-day1equipmenttraining1
  • ses-day1equipmenttraining2
  • ses-day1equipmenttraining3
  • ses-day2equipmenttraining1
  • ses-day2equipmenttraining2
  • ses-preproceduretraining
  • ses-procedure
  • ses-postprocedure
  • ...

Store most, if not all, in one aggregated file

  • phenotype/equipment_training.tsv
    • Where participant_id, session_id, run_id, and day are joint indexes.
    • Yes, day is not a a validate-able column, but you could alternatively create a phenotype/day#_equipment_training.tsv if you want stricter validation support.
    • Remember that run_id is not about tying the data to a particular task's run, but instead to differentiate multiple runs of the same tabular phenotypic test, survey, assessment, questionnaire, or lab result within the same participant_id's session_id.

Example 2: Sleep questionnaires and other questionnaires that administered multiple times un-associated with any imaging sessions.

The answer to example 2 is the appendix example I linked above, and now here.

I hope this helps and let me know if you don't think this resolves your comment. Or if perhaps there's something else we should consider which you can propose?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timdressler

I want to draw attention to a potential issue with the handling repeated usage of a phenotype/ measurement. Currently, if a measurement is conducted more than once, a run_id column MUST be added that "corresponds to an existing run- entity used in a filename(s)" (https://bids-specification--2123.org.readthedocs.build/en/2123/modality-agnostic-files/phenotypic-and-assessment-data.html).

Wowza, good catch! I didn't notice that incorrect set of words there. It currently says:

(run_id is) A run identifier that corresponds to an existing run- entity used in a filename(s). A chronological run number is used when a measurement tool or assessment described by a tabular file was repeated within a session.

What it should say:

(run_id is) A run identifier to differentiate multiple runs of the same measurement tool. A chronological run number is used when a measurement tool or assessment described by a tabular file is repeated within a session.

Thanks! I'll edit that.

For example, I conduct a mood questionnaire 7 times over the course of one session in which multiple tasks (with multiple runs) are conducted.

Does the text edit above of "what it should say" make better sense and resolve the run_id labeling issue?

Comment thread src/appendices/phenotype.md Outdated
Comment thread src/modality-agnostic-files/data-summary-files.md Outdated
Comment thread src/modality-agnostic-files/data-summary-files.md Outdated
Put the phenotypic and assessment data content where it belongs.
Comment thread src/modality-agnostic-files/data-summary-files.md Outdated
Comment thread src/modality-agnostic-files/data-summary-files.md Outdated
Comment thread src/modality-agnostic-files/data-summary-files.md Outdated
Comment thread src/modality-agnostic-files/phenotypic-and-assessment-data.md Outdated
Comment thread src/appendices/phenotype.md Outdated
Comment thread src/appendices/phenotype.md Outdated
Comment thread src/appendices/phenotype.md Outdated
Comment thread src/appendices/phenotype.md Outdated
Comment thread src/appendices/phenotype.md Outdated
@karl-koschutnig

Copy link
Copy Markdown

as @ericearl suggested, I just opened a PR for the prism-idea of survey data structure.

#2404

I would be happy to discuss our approach in detail

Updated the run_id description language to not associate it to a filename, specifically.
@ericearl

ericearl commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Just an FYI to the community after a discussion held between myself, @yarikoptic, and @dmoracze: BEP036 (this PR) will be trying to adopt the more explicit "IndexColumns" solution (issue #2402), instead of just expecting or allowing implicitly for BIDS data curators to put more than one index column into "participants" or "sessions" or "phenotypic and assessment data" files. In fact, BEP036 will be dropping the plan to allow multiple index columns inside the participants and sessions files at all. Support for multiple index columns will at first be restricted to only being in the phenotype/ directory. We also discussed how this idea could be re-used across the BIDS standard in other places later. The full discussion's meeting notes are publicly available here.

P.S. This also resolves @VisLab's Example 1 concern, where participant_id, session_id, run_id, and day all need to be joint indexes for an aggregated tabular file. I think this solution is more optimal all-around for joint indexing in tabular files.

* fix(bst): Address deprecation warnings (bids-standard#2361)

* chore: Set minimum dependencies, use dependency groups

* chore: Add tox.ini to run test suite

* chore(ci): Use tox to run bidsschematools CI

* chore: Update pre-commit excludes

* chore(tox): Enable -Werror to catch incoming warnings

* fix: Address pyparsing deprecation warning

* fix: Only opt in ot pandas 3.0 behaviors if pandas is not 3+

* chore(dependabot): Update uv.locks quarterly

* chore(dependabot): Drop update frequencies to quarterly

* chore(ci): Use FORCE_COLOR, limit token permissions

* chore: Drop Python 3.9 support, test on 3.14

* chore: Add uv.lock for package, update base lock

* chore: Bump schema package to 1.2.2

* chore: Bump schema package to 1.2.3-dev

* fix(ci): Call pre-build script correctly

* [DATALAD RUNCMD] bash -c 'uv lock && (cd tools/schemacode...

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "bash -c 'uv lock && (cd tools/schemacode; uv lock)'",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

* fix: Bad merge

* chore(deps): bump the build-dependencies group across 1 directory with 5 updates (bids-standard#2370)

Bumps the build-dependencies group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [mkdocs-material](https://github.com/squidfunk/mkdocs-material) | `9.7.1` | `9.7.4` |
| [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) | `10.20` | `10.21` |
| [numpy](https://github.com/numpy/numpy) | `2.4.1` | `2.4.3` |
| [myst-nb](https://github.com/executablebooks/myst-nb) | `1.3.0` | `1.4.0` |
| [universal-pathlib](https://github.com/fsspec/universal_pathlib) | `0.3.8` | `0.3.10` |



Updates `mkdocs-material` from 9.7.1 to 9.7.4
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](squidfunk/mkdocs-material@9.7.1...9.7.4)

Updates `pymdown-extensions` from 10.20 to 10.21
- [Release notes](https://github.com/facelessuser/pymdown-extensions/releases)
- [Commits](facelessuser/pymdown-extensions@10.20...10.21)

Updates `numpy` from 2.4.1 to 2.4.3
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v2.4.1...v2.4.3)

Updates `myst-nb` from 1.3.0 to 1.4.0
- [Release notes](https://github.com/executablebooks/myst-nb/releases)
- [Changelog](https://github.com/executablebooks/MyST-NB/blob/main/CHANGELOG.md)
- [Commits](executablebooks/MyST-NB@v1.3.0...v1.4.0)

Updates `universal-pathlib` from 0.3.8 to 0.3.10
- [Release notes](https://github.com/fsspec/universal_pathlib/releases)
- [Changelog](https://github.com/fsspec/universal_pathlib/blob/main/CHANGELOG.md)
- [Commits](fsspec/universal_pathlib@v0.3.8...v0.3.10)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.7.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: build-dependencies
- dependency-name: pymdown-extensions
  dependency-version: '10.21'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: build-dependencies
- dependency-name: numpy
  dependency-version: 2.4.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: build-dependencies
- dependency-name: myst-nb
  dependency-version: 1.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: build-dependencies
- dependency-name: universal-pathlib
  dependency-version: 0.3.10
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: build-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump tornado from 6.5.2 to 6.5.5 (bids-standard#2360)

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.5.2 to 6.5.5.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](tornadoweb/tornado@v6.5.2...v6.5.5)

---
updated-dependencies:
- dependency-name: tornado
  dependency-version: 6.5.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump minimatch from 9.0.5 to 9.0.9 (bids-standard#2355)

Bumps [minimatch](https://github.com/isaacs/minimatch) from 9.0.5 to 9.0.9.
- [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md)
- [Commits](isaacs/minimatch@v9.0.5...v9.0.9)

---
updated-dependencies:
- dependency-name: minimatch
  dependency-version: 9.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump actions/upload-artifact (bids-standard#2369)

Bumps the actions-infrastructure group with 1 update: [actions/upload-artifact](https://github.com/actions/upload-artifact).


Updates `actions/upload-artifact` from 6 to 7
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v6...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions-infrastructure
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump prettier in the node-utilities group (bids-standard#2352)

Bumps the node-utilities group with 1 update: [prettier](https://github.com/prettier/prettier).


Updates `prettier` from 3.8.0 to 3.8.1
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@3.8.0...3.8.1)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.8.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: node-utilities
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump picomatch from 2.3.1 to 2.3.2

Bumps [picomatch](https://github.com/micromatch/picomatch) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](micromatch/picomatch@2.3.1...2.3.2)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 2.3.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump requests from 2.32.5 to 2.33.0 in /tools/schemacode

Bumps [requests](https://github.com/psf/requests) from 2.32.5 to 2.33.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](psf/requests@v2.32.5...v2.33.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix BIDS Starter Kit link in src/index.md (bids-standard#2374)

Updated link for the BIDS Starter Kit to the correct URL.

* chore(deps): bump yaml from 2.8.1 to 2.8.3

Bumps [yaml](https://github.com/eemeli/yaml) from 2.8.1 to 2.8.3.
- [Release notes](https://github.com/eemeli/yaml/releases)
- [Commits](eemeli/yaml@v2.8.1...v2.8.3)

---
updated-dependencies:
- dependency-name: yaml
  dependency-version: 2.8.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump brace-expansion from 2.0.2 to 2.0.3

Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](juliangruber/brace-expansion@v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: brace-expansion
  dependency-version: 2.0.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump pygments from 2.19.2 to 2.20.0

Bumps [pygments](https://github.com/pygments/pygments) from 2.19.2 to 2.20.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](pygments/pygments@2.19.2...2.20.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-version: 2.20.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump pygments from 2.19.2 to 2.20.0 in /tools/schemacode

Bumps [pygments](https://github.com/pygments/pygments) from 2.19.2 to 2.20.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](pygments/pygments@2.19.2...2.20.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-version: 2.20.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump actions/download-artifact

Bumps the actions-infrastructure group with 1 update: [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/download-artifact` from 7 to 8
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions-infrastructure
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump aiohttp from 3.13.3 to 3.13.4

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-version: 3.13.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump lodash from 4.17.23 to 4.18.1

Bumps [lodash](https://github.com/lodash/lodash) from 4.17.23 to 4.18.1.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](lodash/lodash@4.17.23...4.18.1)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.18.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(tox): Fix environment to avoid pre-release tests (bids-standard#2391)

* [pre-commit.ci] pre-commit autoupdate (bids-standard#2381)

updates:
- [github.com/python-jsonschema/check-jsonschema: 0.37.0 → 0.37.1](python-jsonschema/check-jsonschema@0.37.0...0.37.1)
- [github.com/codespell-project/codespell: v2.4.1 → v2.4.2](codespell-project/codespell@v2.4.1...v2.4.2)
- [github.com/pre-commit/mirrors-mypy: v1.19.1 → v1.20.0](pre-commit/mirrors-mypy@v1.19.1...v1.20.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [ENH] Link to example datasets page on the website (bids-standard#2364)

* [FIX] Accept MiscChannelCount in EEG/Motion sidecars; deprecate MISCChannelCount alias (bids-standard#2394)

* schema: accept MiscChannelCount in EEG sidecar; deprecate MISCChannelCount alias

The Recommended-fields rule for EEG sidecars uses MISCChannelCount, but
the validator check rule (MiscChannelCountReq), the spec example in
electroencephalography.md, the MEG and iEEG sidecar rules, and the
legacy validator's JSON schema all use MiscChannelCount. Datasets in the
wild already exist with both spellings.

Make MiscChannelCount the canonical recommended key (matching the rest
of the schema and the docs example) while keeping MISCChannelCount as a
deprecated alias so existing datasets continue to validate.

Closes bids-standard#2393.

* schema: apply MISCChannelCount deprecation to motion sidecar and glossary

Addresses review feedback on bids-standard#2394:

- Apply the same MiscChannelCount (recommended) + MISCChannelCount
  (deprecated) pattern to the motion sidecar rule, matching the EEG
  sidecar change.
- Simplify the EEG sidecar entry to the concise `deprecated` level
  (no per-rule addendum) now that the deprecation note lives with the
  field definition.
- Document the deprecation in the MISCChannelCount description in
  objects/metadata.yaml, so the glossary entry surfaces the canonical
  replacement wherever the field is referenced.

* Fix typos (bids-standard#2399)

* [FIX] Update OSIPI Task force link for ASL lexicon (bids-standard#2396)

* [ENH] Add phenotype and rawbids directories to "study" datasets (bids-standard#2191)

* Suggested modifications to directory layout of the bids-study DatasetType

* fixed minor typos

* removed bids prefix from the DatasetType and subdirs.

* changed rawdata to rawbids directory in the study dataset and updated description in the common principles

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed URL in the JSON example

* Update src/common-principles.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chris Markiewicz <effigies@gmail.com>
Co-authored-by: Chris Markiewicz <markiewicz@stanford.edu>
Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com>

* [ENH] Allow rawbids/ in derivative datasets for the raw BIDS source (bids-standard#2409)

PR bids-standard#2191 added rawbids/ to the study DatasetType but not to derivative,
even though the same convention -- "rawbids/ holds the raw BIDS dataset"
-- is equally applicable when a standalone derivative dataset includes
its raw source.

- Add rawbids as an optional opaque subdirectory of the derivative
  DatasetType in src/schema/rules/directories.yaml.
- Update the derivative example in common-principles.md to use rawbids/
  instead of sourcedata/raw/.
- Clarify that rawbids/ is reserved for raw BIDS datasets in both study
  and derivative cases, and that derivatives of derivatives MUST place
  their source derivative under sourcedata/ (not rawbids/).

Co-authored-by: Claude Code 2.1.116 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ENH] Recommend controlled vocabulary for age Units, clarify that it can be overloaded (bids-standard#2400)

* Clarify that age Units could be overriden and refer to Units in common principles

* Add ISO 8601-based duration units for age and validate in schema

Document in common-principles.md that age Units MAY be overridden
to one of: year, month, week, day, hour, minute, or second (based on
ISO 8601 duration designators).  Add AgeUnits schema check rule that
validates participants.json age Units against the allowed set.

Co-Authored-By: Claude Code 2.1.110 / Claude Opus 4.6 <noreply@anthropic.com>

* Keep compatible (warning, not error) + simplify check

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

* Remove duplicate specification of units in the Description

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>

* Make check operate on participants.tsv not .json

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

---------

Co-authored-by: Claude Code 2.1.110 / Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Chris Markiewicz <effigies@gmail.com>

* [ENH] Allow for institutions to be listed as Authors (bids-standard#2397)

* Allow for institutions to be listed as Authors

It is not uncommon to have datasets which are a truly an institutional effort -- from
data collection planing, acquisition, curation, harmonization etc, where it is a pipeline
to deliver high quality datasets.  For instance we have a number of such contributions from
Allen Institute(s) in DANDI archive.  In DANDI schema we allow for both Person and Organization
entries, and e.g. in https://github.com/dandisets/000020 we have

```yaml
contributor:
- affiliation:
  - identifier: https://ror.org/00dcv1019
    name: Allen Institute for Brain Science
    schemaKey: Affiliation
  email: nathang@alleninstitute.org
  identifier: 0000-0001-8429-4090
  includeInCitation: false
  name: Gouwens, Nathan
  roleName:
  - dcite:ContactPerson
  schemaKey: Person
- contactPoint: []
  identifier: https://ror.org/00dcv1019
  includeInCitation: true
  name: Allen Institute for Brain Science
  roleName: []
  schemaKey: Organization
  url: https://alleninstitute.org
```

so there is an Organization, which is actually the one to cite, although we do
not list any particular roleName, and then responsible ContactPerson (who is
not even listed as an Author) who could be contacted  ATM (but might be a
different person later) about this dandiset.

I have tried to make wording a bit more explicit than just listing Organizations
as a possible entry here, rather to keep it for large efforts.

* Shorten and generalize statement about Authors

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

---------

Co-authored-by: Chris Markiewicz <effigies@gmail.com>
Co-authored-by: Mark Mikkelsen <mark.mikkelsen@gmail.com>

* [FIX] Bump pymdown-extensions to >=10.21.2 to restore code-block rendering (bids-standard#2438)

* [FIX] Bump pymdown-extensions to >=10.21.2 to restore code-block rendering (bids-standard#2421)

Pygments 2.20.0 made `html.escape()` on the formatter's `filename`
option strict, raising `AttributeError` when the value is `None`.
pymdownx.highlight 10.21 (and earlier) passed `filename=title` where
`title` defaults to `None` for code blocks without a title, which
triggered the crash. pymdownx.superfences catches that exception and
silently falls back to inline-`<code>` rendering, so fenced code
blocks inside list items (e.g. the "Plain" examples in
common-principles.md) lost their `<pre>` formatting and showed the
language tag as literal text.

pymdown-extensions 10.21.2 fixes the upstream defect, so require it
(or newer) in both the docs build and bidsschematools[render].

Closes: bids-standard#2421

Co-Authored-By: Claude Code 2.1.160 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [FIX] Regenerate tools/schemacode/uv.lock for pymdown-extensions bump

The previous commit updated tools/schemacode/pyproject.toml but
forgot to refresh its sibling uv.lock. The `latest` tox envs use
the `uv-venv-lock-runner` with `--locked`, so they failed `Setup
test suite` with the pyproject/lockfile mismatch.

Co-Authored-By: Claude Code 2.1.160 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Code 2.1.160 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Chris Markiewicz <markiewicz@stanford.edu>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Remi Gau <remi_gau@hotmail.com>
Co-authored-by: Bru <b.aristimunha@gmail.com>
Co-authored-by: Dimitri Papadopoulos Orfanos <3234522+DimitriPapadopoulos@users.noreply.github.com>
Co-authored-by: Kabilar Gunalan <kabi@mit.edu>
Co-authored-by: Nikhil Bhagwat <nikhil153@users.noreply.github.com>
Co-authored-by: Chris Markiewicz <effigies@gmail.com>
Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-authored-by: Claude Code 2.1.116 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Mark Mikkelsen <mark.mikkelsen@gmail.com>
@sappelhoff
sappelhoff removed their request for review June 14, 2026 09:28
ericearl added 4 commits June 18, 2026 06:38
Completed draft text edits I planned to make for the BEP036 IndexColumns rewrite.
Fixed two broken references to other MD files in the repo. Thanks CI!
Fixed a typo found by codespell. Thanks codespell!

@SamGuay SamGuay left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There you go @ericearl !
(Do we need to wait for #2273 to have this one merge-able?)

Comment thread src/appendices/phenotype.md
Comment thread src/appendices/phenotype.md Outdated
Comment on lines +303 to +304
To read more about the guidelines for tabular phenotypic data and examples,
see the [tabular phenotypic data guidelines appendix](../appendices/phenotype.md).

@SamGuay SamGuay Jun 25, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To read more about the guidelines for tabular phenotypic data and examples,
see the [tabular phenotypic data guidelines appendix](../appendices/phenotype.md).

I don't think we need to repeat it one more time considering the last 3 paragraphs mention the same URL.

Comment thread src/schema/meta/context.yaml Outdated
Comment thread src/schema/rules/checks/dataset.yaml Outdated
Comment thread src/modality-agnostic-files/phenotypic-and-assessment-data.md Outdated
rwblair added 2 commits July 7, 2026 15:21
…sntead of sessions.tsv. Add tool entity to entity ordering. Add additional file rule for phenotype tools files so their entities are loaded into context of js validator.
Comment thread src/schema/rules/checks/phenotype.yaml Outdated
Comment thread src/appendices/phenotype.md
Comment thread src/appendices/phenotype.md
effigies and others added 4 commits July 17, 2026 11:01
1. Added clarity to src/appendices/phenotype.md beginning section
2. Added requirement for participants file to have a JSON in guideline 2
3. Require MeasurementToolMetadata with at least one of TermURL or Description in guideline 3
4. Update IndexColumns definition in phenotypic section
5. Remove sessions_tsv from context.yaml schema
6. Remove accidentally inserted AgeUnits from this PR
7. Remove one phenotype check and update another from sessions to demographics file in src/schema/rules/checks/phenotype.yaml
@ericearl

Copy link
Copy Markdown
Collaborator Author

@rwblair or @effigies: The check_links step failed on a link I don't think I put in there. What would you recommend to get past this error in CI?

URL        `https://www.encodeproject.org/profiles/biosample_type'
Name       `ENCODE Biosample Type'
Parent URL file:///home/circleci/project/site/modality-agnostic-files/data-summary-files.html, line 2318, col 31
Real URL   https://www.encodeproject.org/profiles/biosample_type
Check time 0.516 seconds
Size       2KB
Result     Error: 405 Not Allowed
 1 thread active,    32 links queued, 1229 links in 1262 URLs checked, runtime 46 seconds
 1 thread active,    25 links queued, 1236 links in 1262 URLs checked, runtime 51 seconds
 1 thread active,    14 links queued, 1247 links in 1264 URLs checked, runtime 56 seconds
 1 thread active,     7 links queued, 1254 links in 1264 URLs checked, runtime 1 minute, 1 seconds

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BEP enhancement New feature or request phenotype

Projects

None yet

Development

Successfully merging this pull request may close these issues.